The field of the present invention is confocal microscopy.
Microscopy is used to produce magnified representations of both dynamic and stationary objects or samples; microscopes magnify small things and make them easier to see. There are many different modes of microscopy such as brightfield microscopy, darkfield microscopy, phase contrast microscopy, fluorescence microscopy, reflectance or reflected light microscopy and confocal microscopy. All of these forms of microscopy deliver illumination light in a controlled fashion to the sample and collect as much of the light containing the desired information about the sample as possible. Typically, this is accomplished using Kohler illumination in any of reflectance microscopy, transmission microscopy or epifluorescence microscopy. These methods use appropriately placed diaphragms and lenses to control both the size of the numerical aperture (illumination cone) and the size of the illuminated area of the sample. In Kohler illumination, diaphragms are placed in at least two locations. First, a diaphragm is placed in the conjugate image plane of the sample, a location which permits control of the size of the illuminated area of the sample. Second, a diaphragm is placed in the conjugate image plane of the aperture diaphragm of the objective lens(es) (this location is also a conjugate image plane of the aperture diaphragm of the condenser lens(es)), a location which permits control of the angle(s) of the light illuminating the sample. Typically, any of the diaphragms can be a simple iris (for example, for brightfield microscopy and epillumination fluorescence microscopy), but the diaphragms can also be more complex (for example, in darkfield microscopy, where the diaphragms may comprise cutout rings of different diameters).
An example of a microscope using Kohler illumination is set forth in FIG. 1. In the figure, microscope 2 comprises a light source 4 that emits a plurality of light rays, which have been divided into first light rays 6, second light rays 8 and third light rays 10. The light rays are transmitted along an illumination light path from light source 4 through light source lens 12, adjustable iris field diaphragm 14 and condensor lenses 16. An adjustable iris aperture diaphragm (condenser) 18 can be disposed between upstream and downstream condenser lenses 16. The light then contacts, or impinges upon, sample 20 and then proceeds to pass through objective lenses 22, which objective lenses can comprise an aperture diaphragm (objective) 24 spaced between the objective lenses 22, and then the light rays proceed to a light detector 26. As noted above, the angle of illumination of the sample can be controlled by modulating the light as it passes through conjugate image planes of the aperture diaphragm of the objective lens, which planes can be found, for example, at light source 4 and the upstream aperture diaphragm 18 in FIG. 1, while the location and/or area of illumination of the sample can be controlled by modulating light as it passes through a conjugate image plane of the sample, which plane corresponds to the adjustable iris field diaphragm 14 in FIG. 1.
One preferred form of microscopy is confocal microscopy, in which one or more discreet aperture spots are illuminated in the object plane of the microscope from which transmitted, reflected or fluorescent light is then relayed for observation through conjugate apertures in the image plane. In some embodiments, confocal microscopy can result in spatial resolution about 1.3 times better than the optimum resolution obtainable by conventional light microscopy. See, e.g., U.S. Pat. No. 5,587,832. Additionally, confocal microscopy can reduce the interference of stray, out-of-focus light from an observed specimen above or below the focal plane, and can permit optical sectioning of tissue as well as high-resolution 3-D reconstruction of the tissue. The technique can effectively resolve individual cells and living tissue without staining. Confocal microscopy can be performed using mechanical translation of the specimen with fixed optics, using a fixed specimen and scanning beams manipulated by special rotating aperture disks, or a spatial light modulator (SLM). See U.S. patent application Ser. No. 09/179,185, entitled Apparatus And Methods Relating To Spatially Light Modulated Microscopy; U.S. Pat. Nos. 5,867,251; 4,802,748, 5,067,805, 5,099,363, 5,162,941. The special rotating aperture disks, often called Nipkow disks, typically comprise a plurality of apertures, but only one aperture at a time is used for confocal scanning. Still other known confocal scanning systems have used a laser beam rastered with rotating mirrors to scan a specimen or a laser beam that scans a slit rather than a spot; such slit scanning increases imaging speed relative to rotating aperture disks but slightly degrades resolution. See U.S. Pat. No. 5,587,832. The use of spatial light modulators permits control of either or both of the angle(s) of the light and location of the light, and can provide high speed confocal scanning without the loss of resolution that accompanies slit scanning instead of spot scanning. See U.S. patent application Ser. No. 09/179,185, entitled Apparatus And Methods Relating To Spatially Light Modulated Microscopy; U.S. Pat. No. 5,867,251.
Confocal microscopy, however, does not utilize a significant portion of the light emanating from the spot on the sample that is under investigation, and thus has unnecessarily limited resolution in both the x-y plane (sideways) and in the z-direction (up and down, or depth), and an unnecessarily limited signal to noise ratio. Thus, there has gone unmet a need for improved methods of confocal microscopy that provide enhanced resolution and/or enhanced signal to noise ratio. The present invention provides these and other advantages.
The present invention provides apparatus and methods that improve the depth resolution of confocal microscopy images. The present invention can be applied to all of reflectance microscopy, transmission microscopy and fluorescence microscopy. The present invention comprises utilizing out-of-focus information from within the focal plane of interest (from the x-y direction) and/or from planes above and below the focal plane of interest (from the z-direction). In general, the present invention takes advantage of the observation that in confocal microscopy the intensity of the light emanating from the illumination spot of the sample falls off or decreases in a regular fashion as the distance from the illumination spot increases. For example, the point spread function (PSF) of the emanating light for a confocal, cylindrically symmetric lens system falls off approximately as sinc{circumflex over ( )}2(z) for the singularly illuminated spot, or central illumination pixel, in the vicinity of the focal plane. This PSF in the x-y plane is a function of depth (i.e., of the z-position). The interaction of (a) a reflective surface or other light-emanating surface, such as a fluorescent surface or transmissive surface, and (b) the PSF formed by a confocal microscope, results in xe2x80x9cout-of-focusxe2x80x9d information in, above and below the focal plane; this xe2x80x9cout-of-focusxe2x80x9d information can be measured. By comparing the measurements in the x-y plane, preferably at a plurality of z-positions, one can improve the resolution along each of the x, y, and z-axes. An additional advantage to using the out-of-focus information is that it increases the number of the photons used in the system, thus improving the signal to noise ratio. In addition, incorporation of such information can also improve depth resolution and otherwise help correct for aberrations, such as spherical aberrations or other optical aberrations, in the optical system of a microscope.
Thus, in one aspect the present invention provides confocal microscopes comprising a light detection and analysis system, the system comprising a light detector disposed downstream from a sample in a conjugate image plane of the sample. The detector comprises a central detection pixel positioned to detect and measure in-focus light emanating from a discrete illumination pixel of the sample to provide in-focus data and at least one adjacent detection pixel in an x-y plane relative to the sample that is positioned to independently detect and measure out-of-focus light emanating from the discrete illumination pixel of the sample in the x-y plane. This provides out-of-focus data in the x-y plane. The system further comprises a controller operably connected to the detector and containing computer implemented programming that compiles and combines or convolves the in-focus data and the out-of-focus data to enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data. Generally, the resolution is enhanced in the z-direction by at least about 5% or 10%, preferably by at least 25-100%, and in the x-y plane by at least about 10% and further preferably by at least about 15-40%.
In additional embodiments for this and other aspects of the present invention (unless expressly stated otherwise or clear from the context, all embodiments of the present invention can be mixed and matched), the detector comprises a plurality of adjacent detection pixels that surround the central detection pixel and that independently detect and measure out-of-focus light emanating from the discrete illumination pixel of the sample, and the central detection pixel and the at least one adjacent detection pixel can abut each other. The controller may fit the out-of-focus data in the x-y plane according to a 2D Gaussian distribution or according to other suitable fitting functions. The detector can be movably connected to the sample along a z-axis of the sample such that movement of the detector relative to the sample permits the detector to detect and measure in-focus data from a focal plane of the sample along the z-axis and out-of-focus data from above or below the focal plane along the z-axis and from the x-y direction within each of such planes. The controller can further contain computer implemented programming that compiles and combines the in-focus data from along the z-axis and the out-of-focus data from along the z-axis to enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data from along the z-axis.
In preferred embodiments, the microscope further comprises a spatial light modulator disposed upstream of the sample in a conjugate image plane of the sample and computer implemented programming that causes the spatial light modulator to simultaneously form a plurality of the illumination spots that illuminate a plurality of discrete illumination pixels of the sample and to provide sequential complementary patterns of the spots. The spatial light modulator can be disposed upstream of the sample in a conjugate image plane of the sample and can be operated to selectively alternate between brightfield microscopy and confocal microscopy.
In preferred embodiments, the microscope further comprises a reference mirror disposed in a conjugate image plane of the sample, the reference mirror movably connected to the detector along a z-axis of the mirror such that movement of the reference mirror relative to the detector permits the detector to detect and measure in-focus data from a focal plane of the reference mirror along the z-axis and out-of-focus data from above and below the focal plane along the z-axis, and wherein the controller contains computer implemented programming that compiles the in-focus data from along the z-axis of the reference mirror and the out-of-focus data from along the z-axis of the reference mirror to provide a reference stack of reference mirror images, and combines and convolves or compiles or otherwise compares the reference stack with the measurements of in-focus data and out-of-focus data from the z-axis of the sample, to thereby determine the location of the focal plane of the sample and thus enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data from along the z-axis.
In another aspect, the invention provides a light detection and analysis system, the system comprising a light detector disposed downstream from a sample in a conjugate image plane of the sample, wherein the detector is movably connected to the sample along a z-axis of the sample such that movement of the detector relative to the sample permits the detector to detect and measure in-focus data from a focal plane of the sample along the z-axis and out-of-focus data from above and below the focal plane along the z-axis, the system further comprising a controller operably connected to the detector and containing computer implemented programming that compiles and combines the in-focus data from along the z-axis and the out-of-focus data from along the z-axis to enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data from along the z-axis. Preferably, the detector further comprises a central detection pixel and at least one adjacent detection pixel, and the controller contains computer implemented programming that compiles and combines the in-focus data and the out-of-focus data to enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data.
The microscope can be a reflectance microscope, transmission microscope, fluorescence microscope or other confocal microscope.
In a further aspect, the present invention provides a controller suitable for operable connection to a confocal microscope, wherein the controller comprises a digital light detector disposed downstream from a sample in a conjugate image plane of the sample, the detector comprising a central detection pixel positioned to detect and measure in-focus light emanating from a discrete illumination pixel of the sample to provide in-focus data and at least one adjacent detection pixel in an x-y plane relative to the sample and positioned to independently detect and measure out-of-focus light emanating from the discrete illumination pixel of the sample in the x-y plane to provide out-of-focus data in the x-y plane. Preferably, the controller further contains computer implemented programming that compiles and combines the in-focus data and the out-of-focus data to enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data.
In certain embodiments, the controller fits the out-of-focus data in the x-y plane according to a 2D Gaussian distribution. The detector of the microscope under the control of such controller can be movably connected to the sample along a z-axis of the sample such that movement of the detector relative to the sample permits the detector to detect and measure in-focus data from a focal plane of the sample along the z-axis and out-of-focus data from above and below the focal plane along the z-axis. In such a situation, the controller preferably further contains computer implemented programming that compiles and combines or otherwise compares the in-focus data from along the z-axis and the out-of-focus data from along the z-axis to enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data from along the z-axis.
The controller may also, if desired, fit information where the detector is movably connected along a z-axis to a reference mirror disposed in a conjugate image plane of the sample such that movement of the reference mirror relative to the detector permits the detector to detect and measure in-focus data from a focal plane of the reference mirror along the z-axis and out-of-focus data from above and below the focal plane along the z-axis, wherein the controller further contains computer implemented programming that compiles the in-focus data from along the z-axis of the reference mirror and the out-of-focus data from along the z-axis of the reference mirror to provide a reference stack of reference mirror images, and convolves or otherwise compares the reference stack with the measurements of in-focus data and out-of-focus data from the z-axis of the sample, to thereby determine the location of the focal plane of the sample and thus enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data from along the z-axis.
In an additional further aspect, the present invention provides a confocal microscope comprising means for detecting and measuring out-of-focus light emanating from a discrete illumination pixel of a sample in at least one of an x-y plane and a z-axis to provide a measurement of out-of-focus light, and means for combining the measurement of the out-of-focus light with a measurement of in-focus light emanating from the discrete illumination pixel of the sample, to provide an enhanced resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data. The means for detecting can detect and measure out-of-focus light from both the x-y plane and the z-axis, and the means for combining can combine the measurement of light from both the x-y plane and the z-axis. The microscope can additionally comprise means for providing a reference stack of reference images along the z-axis, and means for convolving the reference stack with the measurements of in-focus data and out-of-focus data from the z-axis of the sample, to thereby determine the location of the focal plane of the sample and thus enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data from along the z-axis.
In yet a further aspect, the present invention provides methods of enhancing resolution of a discrete illumination pixel of a sample using confocal microscopy, comprising detecting and measuring in-focus light emanating from the discrete illumination pixel to provide in-focus data, detecting and measuring out-of-focus light emanating from the discrete illumination pixel in an x-y plane of the sample to provide out-of-focus data in the x-y plane, and compiling and combining the in-focus data and the out-of-focus data to enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data. As above, the out-of-focus data in the x-y plane can be fitted according to a 2D Gaussian distribution or according to other suitable fitting methods.
In preferred embodiments, the method further comprises detecting and measuring in-focus data from a focal plane of the sample along the z-axis of the sample and out-of-focus data from above or below the focal plane along the z-axis. Additionally, the detecting and measuring of out-of-focus data can be from both above and below the focal plane along the z-axis. The methods can also further comprise compiling and combining the in-focus data from along the z-axis and the out-of-focus data from along the z-axis to enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data from along the z-axis, and simultaneously forming a plurality of the illumination spots on the sample to provide a plurality of discrete illumination pixels of the sample, and simultaneously detecting and measuring the light emanating from the plurality of discrete illumination pixels and compiling and combining the in-focus and out-of-focus data from the illumination spots.
In other preferred embodiments, the methods further comprise providing sequential complementary patterns of the illumination spots until the entire surface of the sample has been illuminated. The methods can additionally comprise selectively alternating between brightfield microscopy and confocal microscopy. In some embodiments, the methods can also comprise detecting and measuring in-focus light reflecting from a discrete illumination pixel of a reference mirror disposed in a conjugate image plane of the sample, the in-focus light being from a focal plane of the reference mirror along the z-axis, detecting and measuring out-of-focus light reflecting from the discrete illumination pixel of the reference mirror, the out-of-focus light being from above and below the focal plane along the z-axis, compiling the in-focus data from along the z-axis of the reference mirror and the out-of-focus data from along the z-axis of the reference mirror to provide a reference stack of reference mirror images, convolving the reference stack with the measurements of in-focus data and out-of-focus data from the z-axis of the sample, and determining the location of the focal plane of the sample and thus enhancing resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data from along the z-axis.
In yet another further aspect, the present invention provides methods of enhancing resolution of a discrete illumination pixel of a sample during microscopy comprising detecting and measuring in-focus data from a focal plane of the sample along the z-axis of the sample, detecting and measuring out-of-focus data from above and below the focal plane along the z-axis of the sample, and compiling and combining the in-focus data from along the z-axis and the out-of-focus data from along the z-axis to enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data from along the z-axis.
In some embodiments, the methods can further comprise detecting and measuring out-of-focus light emanating from the discrete illumination pixel in an x-y plane of the sample to provide out-of-focus data in the x-y plane, and compiling and combining the in-focus data and the out-of-focus data in the x-y plane to enhance resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data in the x-y plane, preferably using a 2D Gaussian distribution and simultaneously forming a plurality of the illumination spots on the sample to provide a plurality of discrete illumination pixels of the sample, and simultaneously detecting and measuring the light emanating from the plurality of discrete illumination pixels and compiling and combining the in-focus and out-of-focus data from the illumination spots. Further preferably, the methods comprise providing sequential complementary patterns of the illumination spots until the entire surface of the sample has been illuminated and can include selectively alternating between brightfield microscopy and confocal microscopy.
In further embodiments, the methods can further comprise detecting and measuring in-focus light reflecting from a discrete illumination pixel of a reference mirror disposed in a conjugate image plane of the sample, the in-focus light being from a focal plane of the reference mirror along the z-axis, detecting and measuring out-of-focus light reflecting from the discrete illumination pixel of the reference mirror, the out-of-focus light being from above and below the focal plane along the z-axis, compiling the in-focus data from along the z-axis of the reference mirror and the out-of-focus data from along the z-axis of the reference mirror to provide a reference stack of reference mirror images, convolving the reference stack with the measurements of in-focus data and out-of-focus data from the z-axis of the sample, and determining the location of the focal plane of the sample and thus enhancing resolution of the discrete illumination pixel of the sample when compared to a resolution obtained without using the out-of-focus data from along the z-axis.
These and other aspects, features and embodiments of the present invention are set forth within this application, including the following Detailed Description and attached drawings. In addition, various references are set forth herein, including in the Cross-Reference To Related Applications, that describe in more detail certain apparatus, methods and other information; all such references are incorporated herein by reference in their entirety.